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Abstract 

For a known weak signal in additive white noise, the asymptotic performance of a locally 
optimum processor (LOP) is shown to be given by the Fisher information (FI) of a standardized 
even probability density function (PDF) of noise in three cases: (i) the maximum signal-to- 
noise ratio (SNR) gain for a periodic signal; (ii) the optimal asymptotic relative efficiency 
(ARE) for signal detection; (iii) the best cross-correlation gain (CG) for signal transmission. 
The minimal FI is unity, corresponding to a Gaussian PDF, whereas the FI is certainly larger 
than unity for any non-Gaussian PDFs. In the sense of a realizable LOP, it is found that the 
dichotomous noise PDF possesses an infinite FI for known weak signals perfectly processed by 
the corresponding LOP. The significance of FI lies in that it provides a upper bound for the 
performance of locally optimum processing. 

1 Introduction 

Fisher information (FI) is a measure of the minimum error in estimating an unknown parameter of 
a distribution, and its importance is related to the Cramer-Rao inequality for unbiased estimators 
[Illl]' By introducing a location parameter, the de Bruijn's identity indicates that the fundamental 
quantity of FI is affiliated with the differential entropy of the minimum descriptive complexity of 
a random variable pQ. Furthermore, in known weak signal detection, a locally optimum detector 
(LOD), as an alternative to the Neyman-Pearson detector, has favorable properties for small signal- 
to-noise ratios (SNRs) [0j . With sufficiently large observed data and using the central limit theorem, 
it is demonstrated that the LOD is asymptotically optimum and its asymptotic efficiency is upper 
bounded by the FI of the distribution [2]-[S]. 

However, the fundamental nature of FI is not adequately recognized for processing known weak 
signals. To extend the heuristic studies of PQ-[5], in this paper, we will theoretically demonstrate 
that, for a known weak signal buried in additive white noise, the performance of a locally optimum 
processor (LOP) is completely determined by the FI of a standardized even probability density 
function (PDF) of noise. We show this for three signal processing case studies: (i) the maximum 
SNR gain for a periodic signal; (ii) the asymptotic relative efficiency (ARE) of a LOD for signal 
detection; (iii) the best cross-correlation gain (CG) for an aperidoic (random) signal transmission. 
Moreover, for estimating an unknown parameter of a weak signal, the minimum mean square error 
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of the unbiased estimator can be reduced to a straightforward form expressed by the FI of the 
distribution. The physical significance of FI, resulting from the reciprocal of FI delimiting the 
minimum mean square error of unbiased estimators, provides a upper bound of the performance 
for locally optimum processing. It is well known that a standardized Gaussian PDF has minimum 
FI of unity [2]. As a consequence, for any non-Gaussian noise, it is always possible to achieve 
the performance (SNR gain, ARE or CG) of a LOP larger than unity for the three considered 
situations. In the sense of a realizable LOP, an example of a Gaussian mixture noise PDF is 
investigated [5]. It is found that arbitrarily large FI can be achieved by the corresponding LOP, 
and even when the noise is dichotomous noise associated with infinite FI. 



2 Fisher information measures performances of locally op- 
timum processing 

Since the known weak signal might be periodic or aperiodic, three signal processing cases are 
illustrated for exploring the significance of FI in locally optimum processing. 

2.1 SNR gain for periodic signal processing 

First, consider a static processor with its output 

y(t) = g[x(t)], (l) 

where the nonlinearity g is odd [B] and the input is a signal-plus-noise mixture x(t) = s(t) + z(t). 
The component s(t) is a known weak periodic signal with a maximal amplitude A (0 < \s(t)\ < A) 
and period T. A zero-mean white noise z(t), independent of s(i), has an even (symmetric) PDF 
fz [H E] and the root-mean-square (RMS) amplitude a z (if it exists, or it is a scale parameter.). 
A family of even (symmetric) PDFs is frequently encountered in practical signal processing tasks 
[U IH 03 |B]. In the case of A— >0, we have a Taylor expansion around z at a fixed time t as 

y{t) = g[z + s(t)\ « g(z) + s{t)g'{z), (2) 

where we assume the derivative g'{z) — dg(z)/dz exists for almost all x (similarly hereinafter) [21E] 
. Thus, we have 

E[y(t)} « E[g(z)} + s(t)E[g'(z)]= S (t)E[g'(z)], (3) 

var[y(i)] = E[y 2 (t)} E[y(t)} 2 « E[g 2 (z)}, (4) 

where E[- • •] = /_ • • • f z (z)dz. Here, for an even PDF f z and the odd function g, E[g(z)] = 0, 
E[g(z)g'{z)} = [6 . The higher order infinitesimal s 2 {t){E[g /2 (z)} - E 2 [g'(z)}} is neglected [6], 
resulting in Eq. ((4]). 

The input SNR at x(t) can be defined as the power contained in the spectral line 1/T divided 
by the power contained in the noise background in a small frequency bin AB around 1 /T [7] , this 

_ |( S (t)exp[-z2^/T])| 2 
in " a^ABAt ' l ° j 

with At indicating the time resolution or the sampling period in a discrete-time implementation 
and the temporal average defined as (•••) = y J Q T • • ■ dt 7 . Since s(t) is periodic, y(t) is in general 
be a cyclostationary random signal with period T [7]. Similarly, the output SNR at y(t) is given 

by 

_ |(E[y(t)]expH2^/r])| 2 
^ out ~ (var[y(t)])ABAt ' W 
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with nonstationary expectation E[y(i)] and nonstationary variance v&r[y(t)] [7J. Here, we assume 
the sampling time At -C T and observe the output y(t) for a sufficiently large time interval of NT 
(N > 1) J7J. Substituting Eqs. © and @J| into Eq. © and noting Eq. ©, we have 

J( a (t)expH27rf/r 8 ])| 2 EV(z)] 2 EVM 
«out~ ABAt E[.g 2 (z)] ~ in z E[ 5 2 (^)] ' 

Thus, the output-input SNR gain G# of Eq. © is 

i?o U t ^ oEV(0)] , 
i? in E[5r 2 (z)J 

with the equality occurring as g becomes a LOP, viz. 

g(z)=Cf' z (z)/f z (z)±g opt (z), (8) 

by the Schwarz inequality for a constant C and f' z (z) = df z (z)/dz (2J[HJ[S]- It is noted that the LOP 
g op t of Eq. (JSJ is odd and accords with the above assumption. More interestingly, the expectation 
E [fz{z)/ft{z)] in Eq. © is just the FI I(f z ) of the even noise PDF f z [5]. Furthermore, 
for an even standardized PDF f Zo with zero mean and unity variance a Zo = 1, the scaled noise 
z(t) = a z z (t) has its PDF f z (z) = f ZQ (z/a z )/a z . Since the FI satisfies I{f z ) = I(f Zo )/a 2 z Q]|8], 
the output-input SNR gain Gr of Eq. (JJJ is upper bounded by the FI I(f Zo ) of a standardized PDF 
f Zo (z ), viz. 

G R <I(f Z0 ), (9) 
with equality achieved when g takes the LOP g op t of Eq. © . 



(7) 



2.2 Performance of a LOD for signal detection 

Secondly, we observe a data vector X = {x\, X2, ■ ■ ■ , xn} composed of N observation components 
x n , which might be the white noise z n or the mixture of a signal s n plus white noise z n [2J [5] . 
Consider a generalized correlated detector 

N 

T GC (X) = Y,9(Xn)Sn>J, (10) 

n=l 

with a memory less nonlinearity g and the decision threshold 7 for the hypotheses H± : i„ = s„ + z„, 
otherwise the hypotheses Hq : x n = z n [S]- Also, we assume that the odd function g has zero 
mean under the even PDF f z , i.e. E[p(x)] = 0, and there exists a finite bound A such that 
< \s n \ < A. In the asymptotic case of A — >• and N—>ac, the test statistic Tgc, according 
to the central limit theorem, converges to a Gaussian distribution with mean E[Tgc\Hq] = 
and variance var[Tcc|-ffo] ~ E[g 2 (x)] J2n=i s n under the null hypotheses H . Using Eqs. ([3]) 
and ©, Tqc is asymptotically Gaussian with mean E[Tgo|-Hi] ~ E[g'(a;)] J2 n=1 s 2 and variance 
\ax\Tcc\Hi] = var[TG(7|iJo] under the hypothesis H\ 0[S]. Then, given a false alarm probability 
Pfa , the detection probability of the detector of Eq. (fT0|) is expressed as 

p d = Q[Q- 1 (Pfa)- Vd], (11) 

with Q(x) = exp[— t 2 /2]/^/2n dt and its inverse function Q~ x {x) [2]. It is seen that Pd is a 
monotonic increasing function of the deflection coefficient D 0| given by 



var[d|tf ] ° E[.g 2 ( 2; )] 



s; 



JV iV 



< *(/*) £4 = '(/«>) (12) 



n— 1 n— 1 
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with equality being achieved when g(x) = g op t(x) of Eq. (J5J) [2, 5 . This result indicates that the 
asymptotic optimum detector is the LOD established by the Taylor expansion of the likelihood 
ratio test statistic ln[n„ =1 fz[x n ~ Sn)/IIn=i f*( x n)] ~ Y, n =l 9o P t(x n )s n (C = -1) in terms of 
the generalized Neyman- Pearson lemma [5] . Based on the Bayesian criterion, two hypotheses 
Hq and H\ are endowed with prior probabilities Pq and Pi = 1 — Pq. Similarly, for the weak signal 
s n and the sufficiently large N, the test statistic Tqc in Eq. (jlOl) has Gaussian distribution and 
its performance is evaluated by the error probability [5] 

= /w ! + -r- ! + ivH^r- ^) . (13) 




which is also a monotonically decreasing function of D and has a minimum as D — I(f Zo ) X)n=i s n/ a z 
of Eq. (IT21 for <?(&) = g op t{x) 0[S]. Interestingly, with Dq = Yl n =i s n/ a z (called the signal energy- 
to-noise ratio of the data vector X [2]) achieved by a matched filter as a benchmark [H [5], the 
asymptotic relative efficiency (ARE) 

U = D/D G < I(f Z0 ), (14) 

provides an asymptotic performance improvement of a detector of Eq. (|10[) over the linear matched 
filter [J] when both detectors operate in the same noise environment [21 SI E] - The equality of 
Eq. (fl"4"]) is achieved as g(x) = g op t(x) [5]. 



2.3 Correlation for an aperidoic (random) signal transmission 

Thirdly, we transmit a known weak aperiodic signal s(t) through the nonlinearity g of Eq. ([TJ 
[llj . Here, the signal s(t) is with the average signal variance E[s 2 (t)] = a 2 s <C cr 2 , the zero mean 
E[s(t)] = and the upper bound A (0 < \s(t)\ < A). For example, s(t) can be a sample according 
to a uniformly distributed random signal equally taking values from a bounded interval. The input 
cross-correlation coefficient of s(t) and x(t) = s(t) + z(t) is defined as pi \TT\ 



E[s(t)x(t)} _ Z = ^Zl 

1 cr. 



(15) 
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Using Eqs. ©-(HI), the output cross-correlation coefficient of s(t) and y(t) is given by 

E[s(t)y(t)] ^ asE[g'(z)} 

Then, the cross-correlation gain (CG) G p is given by 

Gp = W{z)] < yiju, a?) 

P Ps, x y/E\g*(z)] ~ 
which has its maximal value as g(z) = g op t{z) of Eq. ([5]). 



2.4 Estimating an unknown parameter of a weak signal 

Finally, for the N observation components x n = s n (9) + z n , we assume the signal s n (9) are with 
an unknown parameter 9. As the upper bound A — > (0 < \s n \ < A), the Cramer- Rao inequality 
indicates that the mean squared error of any unbiased estimator of the parameter 9 is lower bounded 
by the reciprocal of the FI [TJ [5] given by 

N 

1(9) = £E 

n=l 



d In fa (x n 
89 
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/ ds n A 
[ 89' ) 

n— 1 z n— 1 

which indicates that the minimum mean square error of any unbiased estimator is also determined 
by the FI I{f Zo ) of distribution with a location shift, as S n =i(^ L ) /°z i s nxe d- 

Therefore, just as the FI represents the lower bound of the mean squared error of any unbiased 
estimator in signal estimation, the physical significance of the FI I(f Zo ) is that it provides a upper 
bound of the performance for locally optimum processing for the three considered problems. 



df z (z n )/dz 

fz{z n ) 



3 Extreme value of FI of a standardized PDF 



Some interesting questions arise: Which type of noise PDF has a minimal or maximal FI 7(/ Zo ), 
and how large is the extreme value of I(f Zo )t Does the corresponding LOP in Eq. © exist for the 
noise PDF with extreme I(f Za )l These questions will be investigated as follows. 

3.1 Minimal Fisher information of a standardized PDF 



For a standardized even PDF f ZQ , we have 



*(/«,) = E 



E [zl] > E 



f' z M 



fzo(zo) 



(19) 



with E \zq] =& 1 Za =1 and the equality occurring if f z (zo)/ J Zo (zq) = czq for a constant c ^ 0. 
Then, f Zo (zo) = exp \k + czq/2] [2J. In order to be a PDF, c < and exp(fc) is the normalized 
constant [2]. This is a standardized Gaussian PDF f Zo ( z o) = ex P( — Zq/2)/V2tt. Contrarily, any 
standardized non-Gaussian PDF f Zo has the FI I(f Zo ) > 1, which indicates that the performance 
(SNR gain, ARE or CG) is certainly larger than unity via a LOP of Eq. © for processing a known 
weak signal [7] . 



3.2 Maximal Fisher information of a standardized PDF 

A standardized generalized Gaussian noise PDF [5] 

fzM = ci(/3)exp \-c 2 (/3) |z |^ 



(20) 



with c x {p)= {1 l p) ^S?!? + !?? an<I r,( ;:, 



r^(i(i+«) 



r(i(i+gj) 
r(i(i+/3)) 



. The FI J(/ 20 ) of Eq. ([20]) becomes [2] 

4 r[§(i + 0]r (§-£/*) 



(i + /3) a r2 [i(i + /3)] 

with the corresponding normalized LOP [2 

5o P t(x) = l^l^-^/^sign^), 



(21) 



(22) 



where sign(-) is the sign or signum function. The curve of I{f Zo ) versus (3 (cf. Fig. 10.10 of Ref. [2]) 
clearly indicates that, for j3 = 0, I(f Zo ) = 1 is the minimum corresponding to the standardized 
Gaussian PDF. It is also noted that, as j3 — > 3 or — 1, I(f Zo ) ~^ +oo. Is the maximal I(f Zo ) infinite 
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Figure 1: FI I(f Zo ) versus parameter m of the Gaussian mixture noise PDF of Eq. ((26]) . 

for ft = 3 and ft — — 1 or not, and is the corresponding LOP simply implemented? The answer is 
negative, because the LOP of Eq. (|2"2"|) is not realizable as g ov t{x) = ±oo for \x\ > 1 and j3 = — 1. 
When j3 > 1, the LOP of Eq. (l2"2l has a singularity at x = 0. In this sense, an arbitrary large FI 
cannot be reached for the generalized Gaussian noise given in Eq. ([20]) . 
Next, we consider Gaussian mixture noise z(t) with its PDF 



1 



2V2 



-,l^^Wp^ (Z + /i)2 



2e 2 



2c 2 



(23) 



with variance er 2 = /i 2 + e 2 and parameters /i, e > 0. Note that Eq. l|2"3")) has another expression [B] 
as 



with y(z) - 2c 
expressed as 



f z (z)=exp[-y(z)]/V2^, (24) 
z +A1 ln[cosh(^)]. Based on Eq. (|24L the corresponding normalized LOP can be 



g op t(x) = x - n tanh ( — 



fix 



For < m < 1, assume fi = ma z and e 2 = (1 — m 2 )cr 2 , Eq. 
mixture PDF [S] 



(25) 



becomes a standardized Gaussian 



fz (z ) = exp[-y(z )]/ V27r(i - m 2 ), 



(26) 



with y(zo) = — In [cosh( )] ■ The function of FI I(f Zo ) versus m is shown in Fig. Q] 

and I(fz ) can be calculated as (no explicit expression exists) 



Itfz. 



E 



1 — m 



tanhl 



1 — m 2 



1 — m 2 



(27) 



exp 



Interestingly, Fig. [T] shows that, as m = 0, Eq. (|26l) is the standardized Gaussian PDF with 
the FI I(fz ) = 1. While, /(/ zo ) — >• +oo as m — > 1. In Eq. (|2"3"]) . for m -> 1, /i = Vfio z — ► cr z 

i5(z) and is the Dirac delta 

(28) 



and e 2 = (1 — m 2 )a 2 — > 0, the term lim m _>i 
function. Then, Eq. ((23)) becomes 



2e 2 J — 



/ z (z) = -[(5(z-^)+^(z + < 7 z )], 
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which represents the PDF of dichotomous noise z{t). We also note lim^o.M-x^ tanh(^) = sign(x) 
(/j > 0) in Eq. (f^5|) . and the normalized LOP for dichotomous noise z(t) is 



9opt[x) =x- a z sign(x). 



(29) 



Here, the normalized LOP g op t is not continuous at x = 0, but the above analysis is valid for 
processing a known weak signal in dichotomous noise. This point is like the LOP g op t(x) = sign(x) 
for Laplacian noise [2]. 

When z(t) randomly takes two levels — a z and +a z and s(t) is weak compared with z(t) (a z > 
\s(t)\), the signs of input data x{t) = s(t) + z(t) always take the sign of z(t), i.e. sign(a;) = sign(z) 
in Eq. (f2"9"|) . Therefore, the LOP of Eq. (l2"9")l at a fixed time t can be solved as g[x(t)} = x(t) — 
a z sign[x(t)] = s(t) + z(t) — cr z sign[z(<)] = s(t). Moreover, Refs. [JJ [TU] have pointed out that there 
exists a scheme allowing a perfect recovery of s(t) corrupted by dichotomous noise z(t) with the 
PDF of Eq. (|2"5)l . However, a practical difficulty in Eq. (|2"21 is that the RMS a z needs to be known. 
The above analysis indicates that the LOP of Eq. (|29l) can recover the weak signal s(t) perfectly 
as a z > \s(t)\. Thus, according to the optimum performance of the LOP of Eq. @ (Eq. (TH| or 
Eq. (H7J0, the FI /(/*„) = oo contained in the type of PDF of Eq. ([2"g]). as shown in Fig.Q] Using 
Eq. ([2l|i. the FI l(f Zo ) of Eq. ([27]) can be computed as 



E 



( dyjzg) 
V dzo 



E 



d 2 y(z ) 



dzl 



= lim 



>[l-m 2 -m 2 sech 2 (f^)] exp[-y(z )] 



(1 - to 2 ) 2 



dzo =oo, 



(30) 



where lim r 



sech 2 ( yz^s ) = 0, the numerator is the infinitesimal 0(1 — m 2 ) and the denom- 



inator is a higher-order infinitesimal 0((1 — m ) ) in the integral 



4 Conclusions 

In this paper, for a known weak signal in additive white noise, it was theoretically demonstrated 
that the optimum performance of a LOP can be completely determined by the FI of the corre- 
sponding standardized even noise PDF, as illustrated by three signal processing case stuides: (i) 
the maximum SNR gain for a periodic signal; (ii) the optimal ARE for signal detection; (iii) the 
best CG for signal transmission. Thus, our study of the performance of a LOP focused on the 
measure of the FI of a standardized noise PDF. It is well known that the minimal FI is unity for a 
standardized Gaussian noise PDF, and the matched filter is the corresponding optimal processor. 
While, for any non-Gaussian noise, the FI and hence the optimum performance of the LOP is 
certainly larger than unity. Illustratively, we observed that the generalized Gaussian noise PDF 
and the Gaussian mixture noise PDF have an arbitrary large FI. There are some types of noise 
PDF possessing an infinite FI, such as uniform noise and dichotomous noise. However, we argue 
that only if the LOP is practically realizable, can the performance predicted by the FI be reached 
in practice. In this sense, it is found that the dichotomous noise has an infinite FI and also a 
simple LOP structure can be realized in practice. 

Some interesting questions arise. For instance, it is known that for a weak signal already 
corrupted by initial additive white noise, there is usually a LOP that yields the maximal output- 
input gain. Therefore, can the method of adding an extra amount of noise [3] to the initial data 
improve the performance of the updated LOP for the resulting noise PDF? This interesting topic 
will invoke the stochastic resonance phenomenon [HI LT1 [HI HB H2] • Another important question is 
the influence of finite observation time on the performance of locally optimum processing [S] . 
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